Journal manuscript No. 

(will be inserted by the editor) 



Optimality Bounds for a Variational Relaxation of the Image 
Partitioning Problem 

Jan Lellmann • Frank Lenzen • Christoph Schnorr 



o 

(N 
o 

Q 



> 
U 

O 



> 

o 



Received: date / Accepted: date 

Abstract We consider a variational convex relaxation 
of a class of optimal partitioning and multiclass label- 
ing problems, which has recently proven quite success- 
ful and can be seen as a continuous analogue of Lin- 
ear Programming (LP) relaxation methods for finite- 
dimensional problems. While for the latter case several 
optimality bounds are known, to our knowledge no such 
bounds exist in the continuous setting. We provide such 
a bound by analyzing a probabilistic rounding method, 
showing that it is possible to obtain an integral solution 
of the original partitioning problem from a solution of 
the relaxed problem with an a priori upper bound on 
the objective, ensuring the quality of the result from the 
viewpoint of optimization. The approach has a natural 
interpretation as an approximate, multiclass variant of 
the celebrated coarea formula. 
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1 Introduction and Background 

1.1 Convex Relaxations of Partitioning Problems 

In this work, we will be concerned with a class of vari- 
ational problems used in image processing and analysis 
for formulating multiclass image partitioning problems, 
which are of the form 



inf f{u) 



u{x),s{x))dx + / dW{Du) , (1) 

C£:=BY{Q,£) (2) 
= {u e BY{f2y\u{x) e f for a.e. x G /2},(3) 
£:={e\...,e'}. (4) 

The labeling function m : — > R' assigns to each 
point in the image domain f2 := (0, 1)'' a label i G 
I ;= {1, . . . ,1}, which is represented by one of the l- 



dimensional unit vectors e 



, e . Since it is piecewise 



constant and therefore cannot be assumed to be dif- 
ferentiable, the problem is formulated as a free discon- 
tinuity problem in the space BV(17,£) of functions of 
bounded variation; we refer to [2] for a general overview. 

The objective function / consists of a data term and 
a regularizer. The data term is given in terms of the 
function s{x) = {si{x), . . . ,si{x)) S K', and assigns to 
the choice u{x) = the "penalty" Si{x), in the sense 
that 



/ {u{x),s{x))dx = V / 



i{x)dx, 



(5) 



where Hi := ^"-^({e'}) = {x £ {2\u{x) = e*} is the class 
region for label i, i.e., the set of points that are assigned 
the z-th label. The data term generally depends on the 
input data - such as color values of a recorded image, 
depth measurements, or other features - and promotes 
a good fit of the minimizer to the input data. While it 
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is purely local, there are no further restrictions such as 
continuity, convexity etc., therefore it covers many in- 
teresting applications such as segmentation, multi-view 
reconstruction, stitching, and inpainting |24) . 

1.2 Convex Regularizers 

The regularizer is defined by the positively homoge- 
neous, continuous and convex function ^ : R''^' — !• R^q 
acting on the distributional derivative Du of u, and in- 
corporates additional prior knowledge about the "typ- 
ical" appearance of the desired output. For piecewise 
constant u, it can be seen that the definition in ([T]) 
amounts to a weighted penalization of the discontinu- 
ities of u: 



i dW{Du) = 
J u 



(6) 



where Ju is the jump set of u, i.e., the set of points 
where u has well-defined right-hand and left-hand lim- 
its and and (in an infinitesimal sense) jumps 
between the values u'^{x),u~{x) e R' across a hyper- 
plane with normal Vu{x) G R'^, ||t'u(a;)||2 = 1 (see [5] for 
the precise definitions). 

A particular case is to set tf' = (l/-\/2)|| • II2, i.e., the 
scaled Frobenius norm. In this case J[u) is just the 
(scaled) total variation of u, and, since {x) and u~ [x) 
assume values in £ and cannot be equal on the jump 
set Ju, it holds that 

J{u) = ^[ \\u+{x)-u-{x)hdn''~\x), (7) 



V2 



= n''-\Ju). 



(8) 



Therefore, for ^ = ■ II2 the regularizer just 

amounts to penalizing the total length of the inter- 
faces between class regions as measured by the (d— 1)- 
dimensional Hausdorff measure 7i'^~^, which is known 
as uniform metric or Potts regularizer. 

A general regularizer was proposed in |18j . based 
on [S]: Given a metric (distance) d : {1, . . . , l}^ -> R^o 
(not to be confused with the ambient space dimension), 
define 



sup {z,v), 



:={(^>^...,^;')e 



cdx i I 



(9) 
(10) 



V' 



«^||2 < d(z, j)V*, J- e{i,..., /},... 



k=l 

It was then shown that 



therefore in view of Q the corresponding regularizer is 
non-uniform: the boundary between the class regions 
Qi and flj is penalized by its length, multiplied by the 
weight d{i,j) depending on the labels of both regions. 

However, even for the comparably simple regularizer 
([7]), the model ([T]) is a (spatially continuous) combinato- 
rial problem due to the integral nature of the constraint 
set Cf, therefore optimization is nontrivial. In the con- 
text of multiclass image partitioning, a first approach 
can be found in |20) . where the problem was posed in 
a level set-formulation in terms of a labeling function 
)-{l,...,Z}, which is subsequently relaxed to R. 
Then (p is replaced by polynomials in 0, which coin- 
cide with the indicator functions Ui for the case where 
assumes integral values. However, the numerical ap- 
proach involves several nonlinearities and requires to 
solve a sequence of nontrivial subproblems. 

In contrast, representation ([T]) directly suggests a 
more straightforward relaxation to a convex problem: 
replace £ by its convex hull, which is just the unit sim- 
plex in / dimensions, 

A := conv{e\ . . . , e'} (12) 



= {a e R'|a ^ 0,^a, = 1}, 
1=1 



and solve the relaxed problem 
inf f{u) , 



(13) 

C := BY{Q, Al) (14) 
= {u G BY{ny\u{x) e Al for a.e. x € f2} . (15) 
Sparked by a series of papers [^[SlfTT]. recently there 
has been much interest in problems of this form, since 
they - although generally nonsmooth - are convex and 
therefore can be solved to global optimality, e.g., using 
primal-dual techniques. The approach has proven useful 
for a wide range of applications p^lTTlfTOlISS] . 

1.3 Finite-Dimensional vs. Continuous Approaches 

Many of these applications have been tackled before in 
a finite-dimensional setting, where they can be formu- 
lated as combinatorial problems on a grid graph, and 
solved using combinatorial optimization methods such 
as a-expansion and related integer linear programming 
(ILP) methods [HIT5]. These methods have been shown 
to yield an integral labeling u' € Cs with the a priori 
bound 



f{u') < 2^^^2^H£#4/(u^), 



(16) 



d{i,j), 



(11) 



mm^^j d{i,j) 

where Ug is the (unknown) solution of the integral prob- 
lem ([1}. They therefore permit to compute a subopti- 
mal solution to the - originally NP-hard [1] - combina- 
torial problem with an upper bound on the objective. 
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Figure 1 Segmentation of an image into 12 classes using a 
combinatorial method. Left: Input image, Right: Result ob- 
tained by solving a combinatorial discretized problem with 
4-neighborhood. The bottom row shows detailed views of the 
marked parts of the image. The minimizer of the combinato- 
rial problem exhibits blocky artifacts caused by the choice of 
discretization. 




Figure 2 Segmentation obtained by solving a finite- 
differences discretization of the relaxed spatially continuous 
problem. Left: Non-integral solution obtained as a minimizer 
of the discretized relaxed problem. Right: Integral labeling 
obtained by rounding the fractional labels in the solution of 
the relaxed problem to the nearest integral label. The rounded 
result contains almost no structural artifacts. 

No such bound is yet available for methods based on 
the spatially continuous problem (jl3p . 

Despite these strong theoretical and practical re- 
sults available for the finite-dimensional combinatorial 
energies, the function-based, spatially continuous for- 
mulation (dl has several unique advantages: 

— The energy ([1]) is truly isotropic, in the sense that for 
a proper choice of tf' it is invariant under rotation of 
the coordinate system. Pursuing finite-dimensional 
"discretize-first" approaches generally introduces ar- 
tifacts due to the inherent anisotropy, which can 
only be avoided by increasing the neighborhood size, 
thereby reducing sparsity and severely slowing down 
the graph cut-based methods. 

In contrast, properly discretizing the relaxed prob- 
lem p3p and solving it as a convex problem with 
subsequent thresholding yields much better results 
without compromising sparsity (Fig. [T] and [2j [13] ) 
. This can be attributed to the fact that solving the 
discretized problem as a combinatorial problem in 
effect discards much of the information about the 
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problem structure that is contained in the nonlin- 
ear terms of the discretized objective. 

— Present combinatorial optimization methods [4lll5j 
are inherently sequential and difficult to parallelize. 
On the other hand, parallelizing primal-dual meth- 
ods for solving the relaxed problem is straight- 
forward, and GPU implementations have been shown 
to outperform state-of-the-art graph cut methods P5] 

— Analyzing the problem in a fully functional-analytic 
setting gives valuable insight into the problem struc- 
ture, and is of theoretical interest in itself. 

1.4 Optimality Bounds 

However, one possible drawback of the spatially con- 
tinuous approach is that the solution of the relaxed 
problem (|13|) may assume fractional values, i.e., val- 
ues in Ai\£. Therefore, in applications that require a 
true partition of i7, some rounding process is needed in 
order to generate an integral labeling u*. This may in- 
crease the objective, and lead to a suboptimal solution 
of the original problem ([1]). 

The regularizer as defined in Q enjoys the prop- 
erty that it majorizes all other regularizers that can be 
written in integral form and satisfy (jlip . Therefore it 
is in a sense "optimal" , since it introduces as few frac- 
tional solutions as possible. In practice, this forces solu- 
tions of the relaxed problem to assume integral values 
in most points, and rounding is in practice only required 
in small regions. 

However, the rounding step may still increase the 
objective and generate suboptimal integral solutions. 
Therefore the question arises whether this approach al- 
lows to recover "good" integral solutions of the original 
problem ([T]). 

In the following, we are concerned with the ques- 
tion whether it is possible to obtain, using the convex 
relaxation p3p , integral solutions with an upper bound 
on the objective. Specifically, we focus on inequalities 
of the form 

f{u*)^{l + e)f{u}) (17) 

for some constant e > 0, which provide an upper bound 
on the objective of the rownderf integral solution u* with 
respect to the objective of the (unknown) optimal in- 
tegral solution of ([T|). Note that generally it is not 
possible to show that ([T7]) holds for any e > 0. The 
reverse inequality 

^ /("*) (18) 

always holds since u* G and is an optimal integral 
solution. The specific form pT)) can be attributed to the 
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alternative interpretation 



(19) 



whieli provides a bound for the relative gap to the opti- 
mal objective of the combinatorial problem. Such s can 
be obtained a posteriori by actually computing (or ap- 
proximating) u* and a dual feasible point: Assume that 
a feasible primal-dual pair {u,v) € C x T) is known, 
where u approximates u*, and assume that some in- 
tegral feasible u €z has been obtained from w by a 
rounding process. Then the pair (m, v) is feasible as well 
since C C, and we obtain an a posteriori optimal- 
ity bound of the form (fTO|) with respect to the optimal 
integral solution u^: 



/(")~/p(i4) ^ /(t')-/o("£) ^ f(u)-fn{v) / 

/bK) ^ fn{v) fn{v) ^ 



(20) 



However, this requires that the the primal and dual 
objectives / and fn can be accurately evaluated, and 
requires to compute a minimizer of the problem for the 
specific input data, which is generally difficult, espe- 
cially in the spatially continuous formulation. 

In contrast, true a priori bounds do not require 
knowledge of a solution and apply uniformly to all prob- 
lems of a class, irrespective of the particular input. 
When considering rounding methods, one generally has 
to discriminate between 

— deterministic vs. probabilistic methods, and 

— spatially discrete (finite- dimensional) vs. spatially 
continuous methods. 

Most known a priori approximation results only hold 
in the finite-dimensional setting, and are usually proven 
using graph-based pairwise formulations. In contrast, 
we assume an "optimize first" perspective due to the 
reasons outlined in the introduction. Unfortunately, the 
proofs for the finite-dimensional results often rely on 
pointwise arguments that cannot directly be transferred 
to the continuous setting. Deriving similar results for 
continuous problems therefore requires considerable ad- 
ditional work. 



1.5 Contribution and Main Results 

In this work we prove that using the regularizer 
the a priori bound (|16[) can be carried over to the spa- 
tially continuous setting. Preliminary versions of these 
results with excerpts of the proofs have been announced 
as conference proceedings [T^ . We extend these results 
to provide the exact bound (|16p . and supply the full 
proofs. 



As the main result, we show that it is possible to 
construct a rounding method parametrized by a pa- 
rameter ^ €z r, where F is an appropriate parameter 
space: 



R:C X r ^Cs, 

u E C ^ :~ R^(u) G Cf, 



(21) 
(22) 



such that for a suitable probability distribution on F , 
the following theorem holds for the expectation E/(t2) := 
E^/(u-y): 

Theorem 1 Let u e C, s e L^(Qf, s > 0, and let 
'F : M.'^^^ — > R^o be positively homogeneous, convex and 
continuous. Assume there exists a lower bound A; > 
such that, for z ~ {z^ , . . . , z'), 



0. 



(23) 



i=l 



Moreover, assume there exists an upper bound Xu < oo 
such that, for every v G satis fying\v\2 = 1, 



F(v(e 



)')<A„ Vz,je {1,. ..,/}, 



(24) 



Then Alg. 1 (see below) generates an integral labeling 
u (z Cg almost surely, and 



Ef{u) ^ 2^f{u). 



(25) 



Note that A„ ^ A; always holds if both are defined, 
since (|23l) implies, for v with ||i^||2 = 1, 

A„ ^ F{v{e' - e^y) ^ ^(||i.||2 + ||z.||2) = A,. (26) 

The proof of Thm. [1] (Sect. H]) is based on the work of 
Kleinberg and Tardos [12] , which is set in an LP relax- 
ation framework. However their results are restricted in 
that they assume a graph-based representation and ex- 
tensively rely on the finite dimensionality. In contrast, 
our results hold in the continuous setting without as- 
suming a particular problem discretization. 

Theorem[T]guarantees that - in a probabilistic sense 
- the rounding process may only increase the energy in 
a controlled way, with an upper bound depending on ^ . 
An immediate consequence is 

Corollary 1 Under the conditions of Thm. QJ if u* 

minimizes f over C, Ug minimizes f over Cg, and u* 
denotes the output of Alg. 1 applied to u* , then 



lf{u*)^2^f{ul). 



(27) 



Therefore the proposed approach allows to recover, 
from the solution u* of the convex relaxed problem (|13p , 
an approximate integral solution u* of the nonconvex 



Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem 



5 



original problem ^ with an upper bound on the ob- 
jective. 

In particular, for the tight relaxation of the regular- 
izer as in we obtain (cf. Prop. [TU)) 



Xi mmj^jrf(i,jj 



(28) 



which is exactly the same bound as has been proven for 
the combinatorial a-expansion method (|16|) . 

To our knowledge, this is the first bound available 
for the fully spatially convex relaxed problem (fT3)) . Re- 
lated is the work of Olsson et al. [^[^ , where the au- 
thors consider a continuous analogue to the a-expansion 
method known as continuous binary fusion |27| . and 
claim that a bound similar to (jl6p holds for the corre- 
sponding fixed points when using the separable regu- 
larizer 



ndxl 



(29) 



for some A G R''^'^, which implements an anisotropic 
variant of the uniform metric. However, a rigorous proof 
in the BV framework was not given. 

In [3] , the authors propose to solve the problem ((!]) 
by considering the dual problem to ([T^ consisting of I 
coupled maximum- flow problems, which are solved us- 
ing a log-sum-exp smoothing technique and gradient 
descent. In case the dual solution allows to unambigu- 
ously recover an (integral) primal solution, the latter 
is necessarily the unique minimizcr of /, and therefore 
a global integral minimizcr of the combinatorial prob- 
lem ([1]). This provides an a posteriori bound, which ap- 
plies if a dual solution can be computed. While useful 
in practice as a certificate for global optimality, in the 
spatially continuous setting it requires explicit knowl- 
edge of a dual solution, which is rarely available since 
it depends on the regularizer W as well as the input 
data s. 

In contrast, the a priori bound (j27p holds uniformly 
over all problem instances, does not require knowledge 
of any primal or dual solutions and covers also non- 
uniform regularizcrs. 



2 A Probabilistic View of the Coarea Formula 

2.1 The Two-Class Case 

As a motivation for the following sections, we first pro- 
vide a probabilistic interpretation of a tool often used in 
geometric measure theory, the coarea formula (cf. [5]). 
Assuming u' G BV(I2) and u'{x) € [0, 1] for a.e. x G i7, 
the coarea formula states that the total variation of u 



can be represented by summing the boundary lengths 
of its supcr-levclsets: 



TV{u')= f TV(l{„->„})da. 
Jo 



(30) 



Here 1a denotes the characteristic function of a set A, 
i.e., 1a{x) = 1 iS X E a and IaI^) = otherwise. The 
coarea formula provides a connection between prob- 
lem (III) and the relaxation ([T^ in the two-class case, 
where £ = {e^,e^}, u E Cg and mi = 1 — U2'. As noted 
in [16], 

TV(w) = llei - e2|l2 TV(ui) = \/2TV(ui), (31) 
therefore the coarea formula (1301) can be rewritten as 



TY{u) = V2 / TV(l{„,>„j)da (32) 
Jo 

= / T\{e^l{u^>a} + c^'^{m(ia})da (33) 
Jo 



T\{ua)da, 



it„ := e 1 



{til >q} 



(34) 
(35) 



Consequently, the total variation of u can be expressed 
as the mean over the total variations of a set of integral 
labclings {ua G C^rja € [0, 1]}, obtained by rounding u 
at different thresholds a. We now adopt a probabilistic 
view of psp : We regard the mapping 

R : (it, a) eCx [0, 1] ^ Ua e Cg (a.e. a € [0, 1]) (36) 

as a parametrized, deterministic rounding algorithm that 
depends on u and on an additional parameter a. From 
this we obtain a probabilistic (randomized) rounding al- 
gorithm by assuming a to be a uniformly distributed 
random variable. Under these assumptions the coarea 
formula (j35p can be written as 



TV(u) = E„ TV(t2„). 



(37) 



This has the probabilistic interpretation that applying 
the probabilistic rounding to (arbitrary, but fixed) u 
does - in a probabilistic sense, i.e., in the mean - not 
change the objective. It can be shown that this property 
extends to the full functional / in (fO)l : In the two-class 
case, the "coarea-like" property 



(38) 



holds. Functions with property ([55]) are also known as 
levelable functions [51[S] or discrete total variations |5] 
and have been studied in j26j . A well-known implication 
is that if It = u*, i.e., M minimizes the relaxed problem 
(fT^ . then in the two-class case almost every u* = u* 
is an integral minimizcr of the original problem ([1}, 
i.e., the optimality bound ([T7)) holds with e = [3. 
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2.2 The Multi-Class Case and Generalized Coarea 
Formulas 

Generalizing these observations to more than two la- 
bels hinges on a property similar to that holds for 
vector-valued u. In a general setting, the question is 
whether there exist 

— a probability space (-T, /x) , and 

— a parametrized rounding method, i.e., for /x-almost 
every 7 € 

R:C X r ^Cs, (39) 

u e C := Rj{u) e Cs (40) 

satisfying R~f{u') = u' for all u' E Cs, 

such that a "multiclass coarea-like property" (or gen- 
eralized coarea formula) 

f{u) = l^fiu^)df,{^) (41) 

holds. In a probabilistic sense this corresponds to 

f{u) = /(w^)d/i(7) = K^fiu^). (42) 

For I = 2 and ^(x) = \\ ■ II2, ([23 shows that gH) holds 
with 7 = a, r = [0, 1], M = and i? : C X r ^ 
as defined in ((36|l . Unfortunately, property (|37)) is 
intrinsically restricted to the two-class case with TV 
regularizer. 

In the multiclass case, the difficulty lies in provid- 
ing a suitable combination of a probability space {F, fj.) 
and a parametrized rounding step (^,7) 1— > u^. Unfor- 
tunately, obtaining a relation such as ([57]) for the full 
functional ([T]) is unlikely, as it would mean that so- 
lutions to the (after discretization) NP-hard problem 
H]) could be obtained by solving the convex relaxation 
P3|) and subsequent rounding, which can be achieved 
in polynomial time. 

Therefore we restrict ourselves to an approximate 
variant of the generalized coarea formula: 

(1 + e)f{u) ^ f{u-,)dti{-f) = E^fiu.,). (43) 

While is not sufficient to provide a bound on f{u^) 
for particular 7. it permits a probabilistic bound: for 
any minimizer u* of the relaxed problem (jl3p . eq. (j43p 
implies 

E^/(^i;) ^{1 + e)J{u*) ^{1 + e).f{u}), (44) 

i.e., the ratio between the objective of the rounded re- 
laxed solution and the optimal integral solution is bounded 
- in a probabilistic sense - by (1 -t- e). 

In the following sections we construct a suitable 
parametrized rounding method and probability space 
in order to obtain an approximate generalized coarea 
formula of the form (|43p . 



Algorithm 1 Continuous Probabilistic Rounding 

1: u" ^ u, u° ^ n, ^ {1, . . . ,1) em' . 

2: for fc = 1, 2, . . . do 

3: Randomly choose 7*° = (i'°,a'°) £ X x [0, 1] uniformly. 
4: M'' ■<- U'^-'^ n{x e 0\u'^,~'^{x) > a*'}. 

5: u'' -1^ e^''lJ^Jk -^-u''-^ln\M>.■■ 
G■. U'' ^U''-^\M''. 

1 c*' ^ , otherwise . 

8: end for 



3 Probabilistic Rounding for Multiclass Image 
Partitions 

3.1 Approach 

We consider the probabilistic rounding approach based 
on [12] as defined in Alg. 1. 

The algorithm proceeds in a number of phases. At 
each iteration, a label and a threshold 

7^ := (i^a'^-) e r :=Ix [0,1] 

are randomly chosen (step[3|), and label i'^ is assigned to 
all yet unassigned points a; where u^^^{x) > a*^ holds 
(step [5]) . In contrast to the two-class case considered 
above, the randomness is provided by a sequence (7*^) of 
uniformly distributed random variables, i.e., F = (F')^. 

After iteration k, all points in the set U'^ C f] are 
still unassigned, while all points in i7\ U'^ have been as- 
signed an (integral) label in iteration k or in a previous 
iteration. Iteration k-\~l potentially modifies points only 
m the set U''. The variable Cj stores the lowest thresh- 
old a chosen for label j up to and including iteration 
k, and is only required for the proofs. 

While the algorithm is defined using pointwise op- 
erations, it is well-defined in the sense that for fixed 
7, the sequence (u'^), viewed as elements in L^, docs 
not depend on the specific representative of u in its 
equivalence class in L^. The sequences (M*^) and ([/'^) 
depend on the representative, but are unique up to 
negligible sets. 

In an actual implementation, the algorithm could be 
terminated as soon as all points in f2 have been assigned 
a label, i.e., C/'^ = 0. However, in our framework used 
for analysis the algorithm never terminates explicitly. 
Instead, for fixed input u we regard the algorithm as a 
mapping between sequences of parameters (or instances 
of random variables) 7 = (7*^) € F and sequences of 
states (u^), (Ui^) and (cij). We drop the subscript 7 if 
it does not create ambiguities. The elements of the se- 
quence (7'*^^) are independently uniformly distributed, 
and by the Kolmogorov extension theorem [211 Thm. 
2.1.5] there exists a probability space and a stochas- 
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tic process on the set of sequences 7 with compatible 
marginal distributions. 

In order to define the parametrized rounding step 
(u,7) u-y, we observe that once U!^ ~ occurs for 

some k' G N, the sequence {u!^) becomes stationary at {nf, . . . ,nf) ^ Multinomial ( fc; - 



Proof Denote by rij E Nq the number of fc' G {1, . . . , fc} 
such that i'' = j, i.e., the number of times label j was 
selected up to and including the k-th step. Then 



,fc' 



In this case the algorithm may be terminated, with 



output 



,k' . 



Definition 1 Let m e BV(i7)' and / : BV(/?)' ^ M. 
For some 7 € T, if U!^' = in Alg. 1 for some fc' e N, 



we denote 



We define 



(45) 

fiu';'), 3k' eN: u!;' = A e BY(ny, 



'■in—- ^ - y "7 

-00, otherwise. 



We denote by ](u) the corresponding random variable 
induced by assuming 7 to be uniformly distributed on F . 

As indicated above, /(uy) is well-defined: if [/^ =0 
for some (7, fc') then = for all fc" ^ fc'. Instead 
of focusing on local properties of the random sequence 
(it^) as in the proofs for the finite-dimensional case, we 
derive our results directly for the sequence (/(tt^)). In 
particular, we show that the expectation of j{u) over 
all sequences 7 can be bounded according to 



E/(S) = E^/(S^) < (l + e)/(S) 



(46) 



for some e ^ 0, cf. (03]). Consequently, the rounding 
process may only increase the average objective in a 
controlled way. 



3.2 Termination Properties 

Theoretically, the algorithm may produce a sequence 
(m^j) that does not become stationary, or becomes sta- 
tionary with a solution that is not an element of BV(!?)' . 
In Thm. [2] below we show that this happens only with 
zero probability, i.e., almost surely Alg. 1 generates (in a 
finite number of iterations) an integral labeling function 
u-y € . The following two propositions are required for 
the proof. 

Proposition 1 For the sequence (c^) generated by Al- 
gorithm 1, 



(e ' c'^ < 1) ^ 



(47) 



P6{0,1}' V = l ^ ^ / 



holds. In particular, 
P(e ' c'^ < 1) 1. 



(48) 



1 



(49) 



V"' I 

i.e., the probability of a specific instance is 

P((nJ,...,nf)) = I (50) 

I 0, otherwise . 



Therefore, 

P(e^c'= < 1) 



E 

n^,...,n^ 



»(e^c^-<l|(nj,...,nf)) 



P((nJ,...,nf)) 



(51) 



E 



fc! 



!•...• n, 

n'l + ...+n'l=k ^ ' 



•nfM V I 



Since c\, . . . ,c^ < j is a sufficient condition for e^c < 1, 



(e^c^<l|(n^...,nf)). (52) 

; a sufficient condition fc 
we may bound the probability according to 

P(e^c<l)^ 



fc! 



n^! • . . . • ! \ / 

rif + ...+nf=fc 



c)-<ivjel|(n^...,nf)j .(53) 

We now consider the distributions of the components 
of c*"' conditioned on the vector (nf, . . . ,nf). Given 
n'^j, the probability of {rf ^ t} is the probability that 
in each of the steps where label j was selected the 
threshold a was randomly chosen to be at least as large 
as t. For < t < 1, we conclude 

P(c^^ <t|(nj,...,nf)) - V{c'^<t\n'^) (54) 

= 1 - P(cj^ ^ t|7ij^) (55) 

"<^=<' 1 - (1 - t)"^ . (56) 

The above formulation also covers the case n'j = (note 
that we assumed < t < 1). For fixed fc the distribu- 
tions of the c*^ are independent when conditioned on 
(nf 



k ^k\ Therefore we obtain from ([55]) and ((55]> 



(is 



fc! 



nf! 



,nf) (57) 



J5BJ 



V 

zl^ n?! 

r4'-(-...+jif=fc 



fc! 




(58) 
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Expanding the product and swapping the summation 
order, we derive 



f(e^c^ < 1) 



(59) 



kl 



E n 

pe{o,i}' j=i 



1 



= E (-^f' E 



fc! 



(60) 



n?! • . . . • n'i^l 
pe{o,i}' jiJ^+.-.+iif =fc ' 



(61) 



Using the multinomial summation formula, we conclude 

Pie'^c'' < 1) ^ 



E (-1)'"' 

pe{o,i}' 



Pi 



V 



(62) 



which proves (|47p. At (*) the multinomial summation 
formula was invoked. Note that in (p^ the n*^ do not 
occur explicitly anymore. To show the second assertion 
(^5)) . we use the fact that, for any p ^ (0, . . . , 0), qp can 
be bounded by < qp < 1. Therefore 

\fe 



(e^c'^ < 1) > go + E (-1)' ''(«p)' 
pe{o,i}',p#o 

= 1+ J2 i-^f'iip)' 

pe{o,i}',P#o 



(63) 
(64) 

(65) 



which proves (|48 



We now show that Alg. 1 generates a sequence in 
BV(f2)' almost surely. The perimeter of a set A is de- 
fined as the total variation of its characteristic function 
Per(A) := TV(1a). 

Proposition 2 For the sequences [u^), [U^) generated 
by Alg. 1, define 



A:^ f]{7er|Per([/^-) <oo}. 



(66) 



fc=i 



Then 

F{A) = 1. (67) 

//Per(C/^) < oo for all k, then u\ G BV(/7)' for all k 
as well. Moreover, 



\u'' e BY{n)' A PciiU'') < ooVfc e N) = 1, 



(68) 



i.e., the algorithm almost surely generates a sequence 
of BV functions (w'°) and a sequence of sets of finite 
perimeter {U^). 

Proof Wc first show that if Vcy{U^' ) < oo for all k' ^ k, 
then u'' e BV{ny for aU A:' ^ A: as weU. For k = 0, the 
assertion holds, since u° = u e BV(J7)' by assumption. 
For k^ 1, 



fi\A/'' • (69) 

Since Af'^ = U''-^n{n\U''), and U''-, U'"'^ arc assumed 
to have finite perimeter, Af*^ also has finite perimeter. 
Applying ^ Thm. 3.84] together with the boundcdness 
of u*"'^^ and u*^"^ G BV(J?)' by induction then provides 
u'' e BV(i7)'. 

Wc now denote 

I'' r\ Per {u!;)^oo}, (70) 

and the event that the first set with non-finite perimeter 
is encountered at step k e Nq by 

._ jk (^(^r\ i''-^) n . . . n (r \ /") . (71) 

Then 



P(A) 



1 



(72) 



\k=0 



As the sets B'^ arc pairwisc disjoint, and due to the 
countable additivity of the probability measure, we have 



\A)^1-Y, 



(73) 



k=0 



Now U° = n, therefore Per(C/°) = TV(l,7o) = < oo 
and ¥{B°) = 0. For A: ^ 1, we have 

F{B'') ^ P (Per(C/'=) = oo A Per(;7''') < oo Vfc' < fc) 



< P (Pcr{U'') = oo| Per(C/''') < oo Vfc' < fcj 
= P(Per(C/'^-i r\{xe n\u'l,7'^{x) sC a'^}) = oo| 



Per(C/''') < oo Vfc-' < fc-). 



(74) 



By the argument from the beginning of the proof, we 



know that 



,fe-i 



G BV(i7) under the condition on the 



perimeter Per(J7 ), therefore from Thm. 3.40] we 
conclude that Per({a; £ J7|u*i~^(a;) ^ a'^}) is finite for 
£^-a.e. a'^ and all i'^. As the sets of finite perimeter are 
closed under finite intersection, and since the a'' are 
drawn from an uniform distribution, this implies that 

P(Per(i7'=) < oo| PeriU'"^) < oo) = 1. (75) 

Together with ([7^ we arrive at 

P(S'=) = 0. (76) 
Substituting this result into (|73p leads to the assertion, 
P(A) = 1. (77) 
Equation (|68p follows immediately. 
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Using these propositions, wc now formulate the main 
result of this section: Alg. 1 almost surely generates an 
integral labeling that is of bounded variation. 

Theorem 2 Let u e BV(I2)' and f (u) as in Def. U\ 
Then 



P(/(S) < oo) 



1. 



(78) 



Proof The first part is to show that (u^) becomes sta- 
tionary almost surely, i.e.. 



P(3fc e N :[/''■ = 0) = 1. 



(79) 



Assume there exists k such that e^c^ < 1, and assume 
further that ^ 0, i.e., there exists some x G . 
Then Uj{x) ^ for all labels j. But then e^u{x) ^ 
< 1, which is a contradiction to u{x) G Ai. There- 
fore U'' must be empty. From this observation and Prop.[T] 
we conclude, for all fc' € N, 



1 ^ P(3fc e N : = 0) ^ P(e ' c*^' < 1) " 4"" 1, (80) 

which proves ([7^ . 

In order to show that fiu-y) < oo with probability 1, 
it remains to show that the result is almost surely in 
BV(/2)'. A sufficient condition is that almost surely all 
iterates u'^ are elements of BV(J7)', i.e., 



k' ~^QO 



F{u'' e BY{ny V/s e N) = 1. 
This is shown by Prop. [H Then 

P{f{u) < ^) 



(81) 



(82) 



^ ¥{{3k e N : U'' = 1)} A {u'' e BV(i7)' V/s G N}) 
= P({u*= e BV{ny Vfc e N}) (83) 
-P({V/c e N : C/V 0} A {u'' e BV{ny Vk <E N}) 



^ ¥({u'' eBV{ny VfceN})-0 
= 1. 



(84) 
(85) 



Thus P(/(u) < oo) = 1, which proves the assertion. □ 



4 Proof of the Main Theorem 

In order to show the bound and Thm. [1] we first 
need several technical propositions regarding the com- 
position of two BV functions along a set of finite perime- 
ter. Wc denote by (i?)^ and (i?)° the measure-theoretic 
interior and exterior of a set E, see [2], 



P\Q \Bp[x)\ 



= t}, te [0,1]. (86) 



Here Bp{x) denotes the ball with radius p centered in x, 
and \ A\ := £''(A) the Lebesgue content of a set A C R''. 



Proposition 3 Let W be positively homogeneous and 
convex, and satisfy the upper-boundedness condition {24-^ 
Then 

^{iy{z^-zY)^Xu yz\z^-eAi. (87) 
Moreover, there exists a constant C < oo such that 
W{w)^C\\w\\2 yweW, (88) 

W ~{w^{w^\... \w^) e M'^^'l ^ = 0}. (89) 

Proof See appendix. 

Proposition 4 Let E,F C [2^ be -measurable sets. 
Then 

{E n py ^ (E)^ n iF)\ (90) 

Proof See appendix. 

Proposition 5 Let u,v G BV(i7, Ai) and E Q Q such 
that Bcr{E) < oo. Define 

(91) 



w := nil 



vl 



n\E- 



Then w G BV(f2,Z\;)', and 
Dw = Du^{Ef + Dv^{Ef 



TE 



-H''~V(J"£:n J7) , 



(92) 



where and denote the one-sided approximate 

limits of u and v on the reduced boundary J-E, and ue 
is the generalized inner normal of E f^. Moreover, for 
continuous, convex and positively homogeneous W sat- 
isfying the upper-boundedness condition ^24\ l and some 
Borel set Ac [2, 



d^{Dw) ^ / d^{Du) 
A JAniEy 



I 

J At 



dW{Dv) + XuBct{E). (93) 

lAniE)° 
Proof See appendix. 

Propositions Let u,v G BV(i7, Z\/), E C Q such 
that Pcr(i?) < oo, and 

u\(EY = v\(EY C^-a.e. (94) 
Then {Duy{Ey = {Dvy{Ey , and <L'{Duy{Ey = 
'F{Dv)\_{Ey . Ln particular. 



dW{Du) = I d'P{Dv). (95) 

(EY J{Ey 

The result also holds when (i?)^ is replaced by {E)'^ . 
Moreover, the condition ^94\ l is equivalent to 

u\e = v\e C^- a.e. (96) 

Proof See appendix. 

Remark 1 Note that taking the measure-theoretic inte- 
rior {EY is of central importance. The corollary does 
not hold when replacing the integral over (E)^ with the 
integral over E, as can be seen from the example of the 
closed unit ball, i.e., E = Bi{Q), u= \e and v = 1. 
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4.1 Proof of Theorem [T] 



In Sect. 13.21 we have shown that the rounding process 
induced by Alg. 1 is wch-defined in the sense that it re- 
turns an integral solution S BV(i7)' almost surely. 
We now return to proving an upper bound for the ex- 
pectation of f{u) as in the approximate coarea for- 
mula P5)) . We first show that the expectation of the lin- 
ear part (data term) of / is invariant under the round- 
ing process. 

Proposition 7 The sequence (w'^) generated by Alg. 1 
satisfies 



E((m'=,s)) = (w,s) VfceN. 



(97) 



Proof In Alg. 1, instead of step [5] we consider the sim- 
pler update 

u''<-e'''lr fc-i^ fc, -t- ^''"-^Ir k^i. kx- (98) 

This yields exactly the same sequence [u^], since if 
v!lk^{x) > a'' for any a'' ^ 0, then either x e U''^^, 
or u^,^^{x) — 1. In both algorithms, points that are 
assigned a label at some point in the process will 
never be assigned a different label at a later point. This 
is made explicit in Alg. 1 by keeping track of the set 
U'^ of yet unassigned points. In contrast, using the step 
(|98|) . a point may formally be assigned the same label 
multiple times. 

Denote 7' := (7^, . . . , 7'^"-^) and u"^ u^^^. We 
apply induction on k: For fc ^ 1, 



1=1 "'O j=l 



(99) 
da 



{ti7 >"} 



1- V>"}j<"''-^>j'^"- ^^^^^ 

Now we take into account the property [2] Prop. 1.78], 
which is a direct consequence of Fubini's theorem, and 
also used in the proof of the thresholding theorem for 
the two-class case [7]: 



Si{x) ■ l[u,>a}ix)dxda 



Jf2 

= / Si{x)ui{x)dx = {ui,Si). 



(101) 
(102) 



This leads to 

E7(ui;,s) 



= E^, y ^ (s,u7' -I- {v?' ,s) - uj s)) da (103) 



and therefore, using (x) € Ai, 

E^(u^,s) =Ey(M^',s) =E^(u!^-\s). (104) 

Since {vP , s) = (u, s), the assertion follows by induction. 

□ 

Remark 2 Prop. [7] shows that the data term is - in 
the mean - not affected by the probabilistic rounding 
process, i.e., it satisfies an exact coarea-like formula, 
even in the multiclass case. 

Bounding the regularizer is more involved: For 7*^ = 
{i^,a^), define 

U^u := {x e n\uik{x) s; a''}, (105) 

= (C/^.)\ (106) 

= {U'')\ (107) 

As the measure-theoretic interior is invariant under C^- 
negligible modifications, given some fixed sequence 7 
the sequence {V'^) is invariant under £'^-negligible mod- 
ifications of u = u^, i.e., it is uniquely defined when 
viewing u as an element of L^{f2y. Some calculations 
yield 



yk 



n 



n 



k > 1, 



(108) 



{JJ^^C\...C\lJ.,f.)^, fc^2. (109) 
From these observations and Prop. 31 

n...nKyfc, /c ^ 1, (110) 
v^i n (y^2 n . . . n v^k-i) \ 

(y^2 n . .. n )' ^ ^ 2, (111) 



yk-\ \^ yk 



(112) 



k' = \ 



Moreover, since is the measure-theoretic interior 
of , both sets are equal up to an £''-negligiblc set 

(cf. (nzi). 

We now prepare for an induction argument on the 
expectation of the regularizing term when restricted to 
the sets Y^~^ \ . The following proposition provides 
the initial step (fc = 1). 

Proposition 8 Assume that W satisfies the lower- and 
upper-boundedness conditions i23\} and {24^. Then 



E 



/ d^[Du) f d^{Du). 



(113) 
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Proof Denote (i, a) = 7^. Since lu^^ = ^v^i C'^-^.c, 
we have 



"7 = + l^?\V(i,„)"7 



a.e. 



(114) 



Therefore, since = {U°y = {[2^ = i7, 



(115) 



bmce M G BV(f2)', we know that Per(V(j,a)) 

< 00 holds 

for £^-a.e. a and any i [21 Thm. 3.40]. Therefore we 
conclude from Prop. [5] that for £^-a.e. a, 



d^{Du^) ^ A„Per(y(,,„)) + 



(^An.,„,)n(r2\V',.,„,)' 



I(s 



(116) 



(f3\y,,,„,)n(r2\y,.„,) 
Both of the integrals are zero, since De^ ~ and 

(^?\%a))"= (%c))' =%a), (117) 

therefore 

/ dtf'(i?M^) A„Per(%„)). (118) 

Carrying the bound over to the expectation yields 

/ d^{Du^) s^jY^ [ A„ Per(%„) )cia. 

Also, Pcr(V(i_Q,)) = Per(t/(i.Q)) since the perimeter is 
invariant under iC'-negiigible modifications. The asser- 
tion then follows using = J7, = V(^i^a) and the 
coarea formula: 



E. 



(119) 

< tV / A„Per(;7(,,„))da (120) 
' ^=l -^0 

^TV(uO = ^ / ^d||i?^.||2 (121) 



coarca 

T 



123} 2 A r 

<; ^ / d1r{Du). (122) 

We now take care of the induction step for the reg- 
ularizer bound. 



Proposition 9 Let^ satisfy the upper-boundedness con- 
dition \24^. Then, for any k ^ 2, 



F :=E d^{Du) 

a-i; 



-E 



dl'iDu). 



(123) 
(124) 



Proof Define the shifted sequence 7' = (7 )^i by 

Wy := v!;r^ \ v^r^ (125) 
= {v^2n...n v^k-i) \{v^2n...r\V^k) . (126) 

By Prop.[5]we may assume that, under the expectation, 
exists and is an element of BV(i7)'. We denote 7^ = 
(z,a), then V'''^ \V'' = V(,,c«) n due to ^T^. For 
each pair (i,a) we denote by ((i,a),7') the sequence 
obtained by prepending (i, a) to the sequence 7'. Then 
I 

I . 

F 



'■ ,-1 "'O 



dW{Duf^f^,^^)^y^)da. (127) 



Since in the first iteration of the algorithm no points 
in [/(i.c) are assigned a label, '"((i.Q),^') = u-y' holds 
on U(^i^a): and therefore /C'-a.e. on V(i.Q). Therefore we 
may apply Prop. [Hand substitute Du(^(^i a) 7') by Du^r 
in (fHTD: 



E, 



d^iDuy) ]da (128) 



71:/' 

-yV/ ( E7. / ly off (Lilly)) da. (129) 

By definition of the measure-theoretic interior (j86p . the 
indicator function ly^. is bounded from above by the 
density function Ou^^ of U(^i a), 

lv,,„,(x) ^ %,„)(x) Imi , (130) 

which exists H'^~^-a.e. on J7 by [2J Prop. 3.61]. There- 
fore, denoting by Bs{-) the mapping a; G J? i—> ^^(a;), 

F ^ i ^ Ey / hm^ '^'^;L^.^^°^'rf'^P^70^«- 



w , ''NO 



Rearranging the integrals and the limit, which can be 
justified by TY{u-yi) < oo almost surely and dominated 
convergence using ([M)) . we get 



1, 



1 /■ 1 

— E^' lim / , „ , , , 



(131) 



/ / '^{u,{y)iia}dyda\ d^{Duy) . 
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We again apply [H Prop. 1.78] to the two innermost 
integrals (alternatively, use Fubini's theorem), which 
leads to 



F jEy lim / — ^ 



V/ {l-iH{y))dy\d'^{Di 



(132) 



(133) 



Using the fact that u{y) G Ai, this collapses according 
to 



1, 



yEy lim / {I ^ l)d1r{DuY) (134) 

t ''NO 

E^/ /" d^{Du^,) (135) 

d<FiDuy). (136) 



/ - 1 



^E, 



-7' 7' 



Reverting the index shift and using w^/ = Uj concludes 
the proof: 



dl^iDu^) 



(137) 



We are now ready to prove the main result, Thm.[Tl 
as stated in the introduction. 

Proof (Theorem [1]) The fact that the algorithm pro- 
vides u £C£ almost surely follows from Thm. [51 There- 
fore there almost surely exists k' := fc'(7) ^ 1 such 
that ~% and ^ u'^ . On one hand, this implies 



(u^, s) = (u^', s) = limA;^oo(u^;, s) 

almost surely. On the other hand, = {U*' Y 
and therefore 



(138) 



(139) 



k=l 



almost surely. The equality (*) can be shown by induc- 
tion: For the base case fc' = 1, we have = {U^Y = 
{QY = ^- since J7 is the open unit box. For k' ^ 2, 



fe' 



(140) 



fe=i 



fc'-i 



(l/fe'-i \ y'-") u U (y'-^-i \ 1/'=) (141) 
fe=l 

u (142) 

/2\y''^'-\ (143) 



almost surely (cf. (|TT^ ). From ([^5)) and ([1^ we ob- 
tain 



E^/(i2^) =E^ ( lim (u^,s) ) + 

ff:/ 



d^{Du^) 



lim (E^(u^,s)) 



fc— >-oo 



(144) 



(145) 



The first term in (|145p is equal to {u, s) due to Prop.[71 
An induction argument using Prop.|S]and Prop.^lshows 
that the second term can be bounded according to 



00 „ 

VE.^ / d^[Du^) 

k=i Jv^-^W" 



lyk-i\^yk 
k — 1 



k=l 



I 



2 A„ 



2^ / d^iDu) 



(146) 

(147) 
(148) 

(149) 



therefore 

E7/(%) s$ (u,s) +2^ / d^{Du) . 

Since s ^ and Au A; , and therefore the linear term is 
bounded by (m, s) ^ 2(A„/A;)(m, s), this proves the as- 
sertion. Swapping the integral and limit in (jl45p can be 
justified retrospectively by the dominated convergence 
theorem, using ^ (u, s) ^00 and J^^ d^{Du) < 00 
due to the upper-boundedness condition and Prop. |31 

□ 

Corollary [1] (see introduction) follows immediately 

We have demonstrated 



using f{u*) < f{u}), cf. 
that the proposed approach allows to recover, from the 
solution u* of the convex relaxed problem (jl3p . an ap- 
proximate integral solution u* of the nonconvex original 
problem ([T]) with an upper bound on the objective. 
For the specific case W = Wd, we have 

Proposition 10 Let d : E^o be a metric and 

^ = ^d- Then one may set 

Xu ~ max d(i,j) and Xi ~ mindii, j). 

Proof From the remarks in the introduction we obtain 

(cf. m) 

which shows the upper bound. For the lower bound, set 
c := miui^j d{i,j), v'^ := f jp^^ and v := v'{I- jee^). 
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Then V e 2^k)c' since — 1^^112 = Iji''* — if'-'lb ^ c 6 Appendix 
and ve = v'{I — jee7)e = 0. Therefore, for w £ M''^' 



satisfying we = 0, 
^d{w) ^ {w,v) = {w,v') 



c w 



w 



■El 



1 " " ?. — 1 



«^ 2, 



(150) 
(151) 



proving the fower bound. 

Finally, for 'Fd wc obtain the factor 



, All 

'a; 



(152) 



determining the optimality bound, as claimed in the 
introduction ([28|. The bound in ((27)) is the same as 
the known bounds for finite-dimensional metric label- 
ing |12j and a-expansion H], however it extends these 
results to problems on continuous domains for a broad 
class of regularizers. 



Proof (Prop. 0) In order to prove the first assertion 
([87)) . note that the mapping w i-^ 'F{i'w^) is convex, 
therefore it must assume its maximum on the polytope 
Ai — Ai := {z^ — z'^\z^, z^Ai} in a vertex of the poly- 
tope.. Since the polytope Ai — Ai is the difference of 
two polytopes, its vertex set is at most the difference of 



their vertex sets, V := {e* 



,je{l,...,0}.Onthis 



set, the bound W{vw ) ^ Xu holds for w G y due to the 
upper-boundedness condition (|M)) . which proves (|57)) . 

The second equality ([55)) follows from the fact that 
G {b^^ := e'^ {e^ - e'+^Y \\ d, 1 s$ j s$ /-I} is a 

basis of the linear subspace W , satisfying F{V^) ^ A^, 
and F is positively homogeneous and convex, and thus 
subadditive. Specifically, there is a linear transform T : 
W M'^xe-i) such that w = Y.i,k ^'''oLik for a = T{w). 
Then 



(153) 



5 Conclusion 

In this work we considered a method for recovering 
approximate solutions of image partitioning problems 
from solutions of a convex relaxation. We proposed a 
probabilistic rounding method motivated by the finite- 
dimensional framework, and showed that it is possible 
to obtain a priori bounds on the optimality of the in- 
tegral solution obtained by rounding a solution of the 
convex relaxation. 

The obtained bounds are compatible with known 
bounds for the finite-dimensional setting. However, to 
our knowledge, this is the first fully convex approach 
that is both formulated in the spatially continuous set- 
ting and provides a true a priori bound. We showed that 
the approach can also be interpreted as an approximate 
variant of the coarea formula. 

While the results apply to a quite general class of 
regularizers, they are formulated for the homogeneous 
case. Non-homogeneous regularizers constitute an in- 
teresting direction for future work. In particular, such 
regularizers naturally occur when applying convex re- 
laxation techniques [Tll25| in order to solve nonconvex 
variational problems. 

With the increasing computational power, such tech- 
niques have become quite popular recently. For prob- 
lems where the convexity is confined to the data term, 
they permit to find a global minimizer. A proper ex- 
tension of the results outlined in this work may provide 
a way to find good approximate solutions of problems 
where also the regularizer is nonconvex. 



s^f (^^|«,fe|sgn(a,fc)6^'=j (154) 
^Y.\a,k\W {sgnia.kW'') . (155) 

ik 

bmce ([Ml provides tf (±6*'=) ^ A„, we obtain 

>f(H < A„^|a,fe| < A„i|ri|||u.||2 (156) 

ik 

for some suitable operator norm || • || and any w G W. 

Proof (Prop. We prove mutual inclusion: 

" C": From the definition of the measure-theoretic 
interior, 

.eiEnry^mJ-^^^^^l^^i. (157) 

s\o \B5ix)\ 

Since \Bs{x)\ ^ \Bs{x) r\ E\ ^ \Bs{x) r\ E O F\ (and 
vice versa for 1^5(2;) (H -F|), it follows by the "sand- 
wich" criterion that both lima\^o \Bs{x) H ^^[/[^^(a;)! 
and lim^N^o \Bs{x)'^F\/\Bs{x)\ exist and are equal to 1, 
which shows x & E^ f] F^ . 

" D": Assume that x E^ f] F^ . Then 



\Bs{x)f^Ef^F\ 

1 ^ hm sup ,„ , ,| 

■5N0 ^ \Bs{x)\ 



(158) 



^ lim inf 

(5\0 



\Bs(,x)r\EnF\ 
\Bdx)\ 



(159) 



. , \Bs{x) f^E\ + \Bs f^F\- \Bs f^{E\JF)\ 
lim mi 



(S\,0 



\B8{x)\ 
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We obtain equality, 
l>lin.inf.l^^(^)^^^^l 



5\0 \B5{x)\ 

>lin.infM^ + l,,inf.l^^(-^)^^l 



lim inf ( — 



\Bsn{EUF )\' 

mx)\ 



„ \Bsr\(EUF)\ , 

2 - lim sup ' ,, ^' ^ 1, 



(160) 

(161) 
(162) 



additive on mutually singular Radon measures /i, v, i.e., 
if |/^|_L|i^|, then 



(172) 



for any Borel set B <Z Q. Substituting Dw in (|17ip ac- 
cording to ([M)) and using the fact that the three mea- 
sures that form Dw in (|92p are mutually singular, the 
additivity property (|172p leads to the remaining asser- 
tion, 



from which we conclude that 

\Bs{x)f^Ef^F\ . , \Bs{x)f^Ef^F\ ^ 

hm sup ,^ , = hm ml ,^ , ~ 1 , 

i.e., x€ (£;ni^)\ 

Proof (Prop. (3) First note that 



^ sup{||u'J^^(a;) — ti;jr^(a;)||2|a; e J^i? n i?} 

H'^-^{FEr\n) 

(*) 

^ sup{||«;(a;)-«;(y)||2|x,y e X2}-TV(li=;) (165) 



(164) 



(**) r- 

^ V2TV(1b) 
= V2Pcr(£') < 00. 



(166) 
(167) 

The inequality (*) is a consequence of the definition of 
and [H Thm. 3.59], and (**) follows directly from 
w{x),w{y) € Al. The upper bound p67|) permits ap- 
plying [2 Thm. 3.84] on w, which provides w G BV(i7)' 
and dMl)- Due to [2j Prop. 3.61], the sets {Ef, {Ef and 
J^ii' form a (pairwise disjoint) partition of Q, up to an 
W'-'^-zevo set. Moreover, since ^{Du) < < H''"^ 
by construction, we have, for some Borel set A, 



1'{Dw) (168) 
d<F{Dw) + / d9{Dw) + (169) 



If 



d^{Dw) + / d^{Dw) 
Xud-W^-^ 



^ / d1'{Dw) 
JAn(EY 

Xu PeriE). 



An{E)0 



d^{Dw) 



(170) 



(171) 



The inequality (**) holds due to the upper boundedness 
and Prop. [31 From [H Prop. 2.37] we obtain that ^ is 



d^{Dw) ^ 



d^{Du) 



An{E)^ 



An(Ey> 



(173) 

d^{Dv) + Xu FeriE) . 



Proof (Prop. 0) We first show ([M)) . It sufiices to show 
that 



(163) {a; e (Ey ^ X eE} for C^-Si.c. x € H. 



(174) 



This can be seen by considering the precise representa- 
tive 1b of 1^; [H Def. 3.63]: Starting with the definition. 



s\o \Bsix)\ 



the fact that lim^x^o '"g"^^^^^ = 1 implies 



x e (E)'^ ^ lim 
<4> lim 



\f2nBs{x)\ _ 

\{n\E)nBs{x)\ 



\Bsix) 



1 



s\o \Bs{x)\ Js,(^) 
^ 1e{x) = 1. 



|l£-l|dy = 



(175) 

(176) 
(177) 
(178) 



Substituting E hy f2 \ E, the same equivalence shows 
that X G {Ef ^ la\E{x) = 1 <^ 1e{x) = 0. As 
£'^(^7 \ {{Ef U {EY)) = 0,J^his shows that lei = hs 
C^-a.c. Using the fact that 1^; = 1^; ^ Prop. 3.64], we 
conclude that 1(_e)i = Ie C^-'^.c, which proves (|174p 
and therefore the assertion ([95)1 . 

Since the measure-theoretic interior (E)^ is defined 
over /C^-intcgrals, it is invariant under yC'-negligible mod- 
ifications of E. Together with (|174p this implies 

((i?)i)i = {E)\ HEf = TE, {{Eyr = [Ef . (179) 

To show the relation {Du)^{Ey = {Dv)i_{Ey , consider 

Du^iEy = D {ln\(Eyu + l(£)iw) ^{EY (180) 



= D (lr2\(B)iw 



v).{E)\ 



(181) 



The equality (*) holds due to the assumption and 
due to the fact that Df = Dg ii f — g C^-a.e. (see, 
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e-g-j [21 Prop. 3.2]). We continue from p81[) via 

Du^{Ef (182) 

(183) 



^•^-^(^(s)! n /?)}l(s)1 



{Du^iEf + Dv^iEY) l{E)^ + (184) 
(ly^EV {u%^,-v^^,Y 'H''-\{FEr\n))^{EY 
= Du^{{Efr\{EY)+Dv^{{EYr\{EY)+ (185) 

{u%^,-v-^^,Y 'H'-K{FEC\nc\{Ef) 
= Dv^{Ef. (186) 

Therefore Du^{EY ^ Dv^{EY . Then, 
IfiDuyiEY = ^{Du^{EY + 

i:'uL(r2\ (s)1))l(s)1 (187) 

If (L'ul(S)I) l(S)1 + 

If (i?UL(r2\ (£;)!)) l(^)1. (188) 

In the equahty (*) we used the additivity of ^ on mutu- 
ally singular Radon measures [U Prop. 2.37]. By defini- 
tion of the total variation, = \ij\^A holds for any 
measure therefore \Du^{n\{Ef )\ = \Du\^{n\{EY) 
and \Du^{[2 \ {EY)\{{EY) 0, which together with 
(again by definition) 'f'(/i) ^ implies that the second 
term in (|188p vanishes. Since all observations equally 
hold for V instead of u, we conclude 



l^iDuYiEf ^ W{Du^{E)^y{EY 

^ nDv^{Efy{Ef 
= WiDvyiE)\ 

Equation ([95)1 follows immediately. 



(189) 

(190) 
(191) 



References 

1. Alberti, G., Bouchitte, G., Dal Maso, G.: The cahbra- 
tion method for the Mumford-Shah functional and free- 
discontinuity problems. Calc. Var. Part. Diff. Eq. 16(3), 
299-333 (2003) 

2. Ambrosio, L., Fusco, N., Pallara, D.: Functions of 
Bounded Variation and Free Discontinuity Problems. 
Clarendon Press (2000) 

3. Bae, E., Yuan, J., Tai, X.C.: Global minimization for con- 
tinuous multiphase partitioning problems using a dual 
approach. Int. J. Comp. Vis 92, 112-129 (2011) 

4. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate en- 
ergy minimization via graph cuts. Patt. Anal. Mach. In- 
tell. 23(11), 1222-1239 (2001) 

5. ChamboUe, A., Cremers, D., Pock, T.: A convex approach 
for computing minimal partitions. Tech. Rep. 649, Ecole 
Polytechnique CMAP (2008) 

6. ChamboUe, A., Darbon, J.: On total variation minimiza- 
tion and surface evolution using parametric maximum 
flows. Int. J. Comp. Vis 84, 288-307 (2009) 



7. Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for 
finding global minimizers of image segmentation and de- 
noising models. J. Appl. Math. 66(5), 1632-1648 (2006) 

8. Darbon, J., Sigelle, M.: Image restoration with discrete 
constrained total variation part I: Fast and exact opti- 
mization. J. Math. Imaging Vis. 26(3), 261-276 (2006) 

9. Darbon, J., Sigelle, M.: Image restoration with discrete 
constrained total variation part II: Levelable functions, 
convex priors and non-convex cases. J. Math. Imag- 
ing Vis. 26(3), 277-291 (2006) 

10. Delaunoy, A., Fundana, K., Prados, E., Heyden, A.: 
Convex multi-region segmentation on manifolds. In: 
Int. Conf. Comp. Vis. (2009) 

11. Goldstein, T., Bresson, X., Osher, S.: Global minimiza- 
tion of Markov random field with applications to optical 
fiow. CAM Report 09-77, UCLA (2009) 

12. Kleinberg, J.M., Tardos, E.: Approximation algorithms 
for classification problems with pairwise relationships: 
Metric labeling and Markov random fields. In: 
Found. Comp. Sci., pp. 14-23 (1999) 

13. Klodt, M., Schoenemann, T., Kolev, K., Schikora, M., 
Cremers, D.: An experimental comparison of discrete 
and continuous shape optimization methods. In: Eu- 
rop. Conf. Comp. Vis. Marseille, France (2008) 

14. Kolev, K., Klodt, M., Brox, T., Cremers, D.: Continu- 
ous global optimization in multiview 3d reconstruction. 
Int. J. Comp. Vis. 84(1) (2009) 

15. Komodakis, N., Tziritas, G.: Approximate label- 
ing via graph cuts based on linear programming. 
Patt. Anal. Mach. Intell. 29(8), 1436-1453 (2007) 

16. Lellmann, J., Becker, F., Schnorr, C: Convex opti- 
mization for multi-class image labeling with a novel 
family of total variation based regularizers. In: 
Int. Conf. Comp. Vis. (2009) 

17. Lellmann, J., Kappes, J., Yuan, J., Becker, F., Schnorr, 
C: Convex multi-class image labeling by simplex- 
constrained total variation. In: Scale Space and 
Var. Meth., LNCS, vol. 5567, pp. 150-162 (2009) 

18. Lellmann, J., Lenzen, F., Schnorr, C: Optimality bounds 
for a variational relaxation of the image partitioning 
problem. In: Energy Min. Meth. Comp. Vis. Patt. 
Recogn. (2011) 

19. Lellmann, J., Schnorr, C: Continuous multiclass labeling 
approaches and algorithms. SIAM J. Imaging Sci. p. In 
press. (2011) 

20. Lysaker, M., Tai, X.C.: Iterative image restoration com- 
bining total variation minimization and a second-order 
functional. Int. J. Comp. Vis. 66(1), 5-18 (2006) 

21. 0ksendal, B.: Stochastic Differential Equations, 6th edn. 
Springer (2003) 

22. Olsson, C: Global optimization in computer vision: Con- 
vexity, cuts and approximation algorithms. Ph.D. thesis, 
Lund Univ. (2009) 

23. Olsson, C, Byrod, M., Overgaard, N.C., Kahl, F.: Ex- 
tending continuous cuts: Anisotropic metrics and expan- 
sion moves. In: Int. Conf. Comp. Vis. (2009) 

24. Paragios, N., Chen, Y., Faugeras, O. (eds.): The Hand- 
book of Mathematical Models in Computer Vision. 
Springer (2006) 

25. Pock, T., Cremers, D., Bischof, H., ChamboUe, A.: Global 
solutions of variational models with convex regulariza- 
tion. J. Imaging Sci. 3(4), 1122-1145 (2010) 

26. Strandmark, P., Kahl, F., Overgaard, N.C.: Op- 
timizing parametric total variation models. In: 
Int. Conf. Comp. Vis. (2009) 

27. Trobin, W., Pock, T., Cremers, D., Bischof, H.: Contin- 
uous energy minimization by repeated binary fusion. In: 
Europ. Conf. Comp. Vis., vol. 4, pp. 667-690 (2008) 



16 



Jan Lellmann et al. 



28. Yuan, J., Bae, E., Tai, X.C., Boykov, Y.: A contin- 
uous max-flow approach to potts model. In: Eu- 
rop. Conf. Comp. Vis., pp. 379-392 (2010) 

29. Zach, C, Gallup, D., Frahm, J.M., Niethammer, M.: Fast 
global labeling for real-time stereo using multiple plane 
sweeps. In: Vis. Mod. Vis (2008) 



